The present invention relates to multi-touch technology, and more specifically, to a multi-touch method and apparatus for general-purpose computers, and a portable terminal device.
The multi-touch technique is a technique allowing computer users to control graphic interfaces with multiple fingers simultaneously. With the prevalence of multi-touch capable smart phones, gesture recognition based on multi-touch control has gradually become a widely accepted human-machine interaction technique.
Multi-touch input must rely on multi-touch control devices, generally, multi-touch pads or touch screens supporting multi-touch input. However, such devices are not very popular nowadays, which commonly only present in high-end electronic devices, such as notebook computers, or need to be purchased separately. Many home or commercial desktop computers and notebook computers do not support multi-touch in terms of hardware, thus the applications of multi-touch input technology on general purpose computers are limited.
In order to develop and apply software systems supporting multi-touch input in the absence of multi-touch devices, many techniques of multi-touch input simulation have been developed. For example, a multi-touch input simulation software covers the window of target software which supports multi-touch input, and one or several points are clicked with a mouse device on the target software as fixed input points, then a moveable input pointer is simulated with the mouse device. It has also been done to simulate two input points by two mouse devices. The number of touch points that can be simulated by such method is very limited, with a huge difference between this input manner and that of multi-touch input operations, and an unfavorable effect.
Gesture recognition based on machine vision has emerged recently, in which gestures are captured with a camera, and interaction semantics of gestures are determined through calculating correlation coefficients between motion traces of gestures and predetermined templates. Existing methods of recognizing fingers and simulating multiple touches through a camera require a physical surface with checkerboard lattices as a simulated touch pad. Images of motions of fingers are taken by a camera supporting wireless communication, and then transmitted to a computer through wireless communication for analysis. Multi-touch inputs of users are acquired through camera auto-calibration, pose learning before using, image recognition in use, and the like methods.
However, such camera-based gesture recognition requires additional accessory devices, such as a physical surface with checkerboard lattices, a wireless camera added for capturing finger movements, an embedded transmission device, and connection to a computer through a USB wireless receiver. However, these requirements apparently limit the applications of such technique. In addition, such methods are unable to solve problems about the start and termination of inputs favorably. When fingers are present in images, users may possibly have not prepared to start or withdrawn fingers after their operations. It may cause a great inconvenience to the operations of users if it is unable to accurately distinguish the start and termination of inputs, or even lead to operation confusion, losing its practical value as a result.
Therefore, there is a need for a multi-touch technique applicable for general purpose computers, capable of overcoming defects in the prior art, which, on common personal computers and on the premise of without additional hardware devices, can support multi-touch input of users in a simple and natural manner as approximate as possible to that of practical multi-touch device operations.